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Abstract 

Background: Motor learning is central to domains such as sports and rehabilitation; however, often terminologies are 
insufficiently uniform to allow effective sharing of experience or translation of knowledge. A study using a Delphi technique 
was conducted to ascertain level of agreement between experts from different motor learning domains (i.e., therapists, 
coaches, researchers) with respect to definitions and descriptions of a fundamental conceptual distinction within motor 
learning, namely implicit and explicit motor learning. 

Methods: A Delphi technique was embedded in multiple rounds of a survey designed to collect and aggregate informed 
opinions of 49 international respondents with expertise related to motor learning. The survey was administered via an 
online survey program and accompanied by feedback after each round. Consensus was considered to be reached if >70% 
of the experts agreed on a topic. 

Results: Consensus was reached with respect to definitions of implicit and explicit motor learning, and seven common 
primary intervention strategies were identified in the context of implicit and explicit motor learning. Consensus was not 
reached with respect to whether the strategies promote implicit or explicit forms of learning. 

Discussion: The definitions and descriptions agreed upon may aid translation and transfer of knowledge between domains 
in the field of motor learning. Empirical and clinical research is required to confirm the accuracy of the definitions and to 
explore the feasibility of the strategies that were identified in research, everyday practice and education. 
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Background 

Motor learning is a central issue in sports, but has recently 
received increased attention in the context of rehabilitation [1—6]. 
A departure point for explanatory models of motor learning, in a 
variety of healthy and rehabilitation target groups, is the broad 
distinction between conscious and non-conscious attributes of the 
motor learning process. The distinction is often delimited by an 
implicit-explicit conceptualisation first made popular in cognitive 
psychology [7], which proposes that implicit motor learning 
targets more non-conscious attributes of the motor learning 



process, whereas explicit motor learning targets more conscious 
attributes of the motor learning process [8,9]. 

Although investigation of implicit and explicit forms of motor 
learning has become common-place in recent years, particularly in 
the sport-related motor literature [9,10], for the most part, 
terminology has been insufficiently uniform. This has raised a 
barrier that hampers exchange of knowledge within and between 
fundamental domains of research and practical target groups. 

For example, in describing or defining implicit and explicit 
(motor) learning, independent research groups have focused on the 
type of knowledge accrued during the learning process, e.g. [8], 
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Table 1. Content and structure of the survey rounds. 





Round Content 


Questions 


Answering options 


1 Definition of implicit and explicit learning (part 1): 

Experts were provided with attributes used in the 
literature to define or describe implicit and explicit 


The definition of implicit motor learning should 
in your opinion definitely contain the following 
attributes: 


multiple choice, more answers possible, 
see Results section for an overview of 
attributes provided 



motor learning. The following questions were asked: 





The definition of explicit motor learning should 
in your opinion definitely contain the following 
attributes: 


multiple choice, more answers possible, 
see Results section for an overview of 
attributes provided 




If you are aware of a definition of 
explicit/implicit motor learning from the 
literature, with which you can agree (best 
option), please give a citation of this definition 
in the box below. Please include the reference. 


open comment box 


Identification and description of strategies 
promoting motor learning and their classification 
(part 2): A list of motor learning strategies was 
provided together with a description based on the 
literature. For each strategy the following questions 
were asked (see Results section for an overview of 
strategies provided): 


Do you know the strategy? 


dichotomous choice: yes/no, experts 
who agreed were referred to the next 
question, expert who did not agree 
were referred to the next strategy in the 
list 




Do you agree with the provided description? 
If not, please indicate what is missing or 
incorrect and/or provide your ideal description. 


dichotomous choice: yes/no and open 
box, see results section for an overview 
the provided descriptions 




How would you classify the strategy? 


multiple choice, only one answer 
possible, see results section for the 
answering categories provided 




Have you used the strategy before (research 
or practice)? 


dichotomous choice: yes/no 




Can you give an example of how you would 
apply this strategy in practice or research? 


open comment box 


2 Confirmation of results of Part 1 from Round 1: 

Based on the results of part 1, the definitions of implicit 
and explicit motor learning were provided. Separately 
for the definitions of implicit and explicit motor learning 
the experts were asked: 


Do you, in general, agree with the definition? 


dichotomous choice: yes/no 




Please state any comments or additional 
information in the box. 


open comment box 


Confirmation of results Part 2 from Round 1: 

Only the best known-strategies were taken into account 
in this round. For each strategy the experts were asked: 


Do you, in general, agree with the modified 
description? 


dichotomous choice: yes/no 



doi:1 0.1 371 /journal.pone.01 00227.t001 



the amount of attention or awareness needed to learn, e.g. [1 1— 
13], or the way the results of the learning process are measured, 
e.g. [11,14]. The terms implicit and explicit are sometimes also 
used to refer to the underlying memory systems that are involved, 
e.g. [15], as synonyms associated with declarative and procedural 
knowledge, e.g. [12] or to describe the actual application of 
learning in practice, e.g. [16]. However, there is some overlap in 
underlying conceptualisations of implicit and explicit motor 
learning. For example, in many definitions and descriptions 
explicit motor learning is associated with conscious cognitive 
processes, whereas implicit learning is associated with non- 
conscious cognitive processes, e.g. [8,17]. The lack of agreement 
sometimes results in different, or even conflicting, applications of 
implicit and explicit learning in study paradigms, clinical practice 
and education. 

Intervention strategies, such as discovery learning, analogy 
learning and errorless learning, have been used to shape the motor 
learning process in clinical or non-clinical studies within different 



target groups [18-20]. In general, intervention strategies that lead 
to high conscious awareness of how the motor behavior is 
accomplished are applied to promote explicit motor learning, 
whereas intervention strategies resulting in low conscious aware- 
ness of how the motor behavior is accomplished are applied to 
promote implicit motor learning [8,9]. The theoretical underpin- 
ning for this implicit/explicit distinction proposes that motor 
learning is a process in which solutions to the motor problem are 
discovered either explicitly through a process of hypothesis testing 
made possible by the human ability to temporarily manipulate and 
store information consciously in working memory, or implicidy 
through a process of discovery that does not rely on conscious 
manipulation and storage of information by working memory 
[8,9,21]. For example, reducing the amount of errors made during 
the motor learning process (errorless or error-reduced learning) is 
thought to moderate the need for hypothesis testing about possible 
motor solutions, which minimizes working memory involvement 
in movement and promotes implicit motor learning [19,22]. 
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Researchers identified 
through literature 

N=66 

sport/healthy: n=11 
rehabilitation: n=55 



Researchers identified 
through networks 

N=36 

sport/healthy: n=21 
rehabilitation: n=15 



Researchers identified 
through networks 

N=12 



Recruitment round 
N= 114 invited 



Excluded 

n=4 no working mail address/retired 

n=7 refused to participate 
n=1 unavailable during study period 



Agreed to participate: 

n=39 



Recommended by invited experts: 

n=49 

Invited to recruitment round: 

n=41 

n=2 recommended twice 
n=6 already invited 



Final expert panel 
N=55 



Agreed to participate: 
N=16 

n= 8 Researcher 
n= 8 Therapist/coach 



Round 1: 

n=49 

n=1 did not complete round one, 
but only round 2 



withdrawal: n= 1 
non-response: n= 5 



Round 2: 

n=44 



withdrawal: n=2 
non-response: n=9 



Figure 1. Recruitment and compilation of experts. 

doi:1 0.1 371 /journal.pone.01 00227.g001 



Further, it has been argued that learning a motor task while 
performing a concurrent cognitive task (dual task learning) 
prevents working memory from temporarily storing conscious 
information related to motor solutions because working memory 
must engage in completing the cognitive task. Thus, the motor 



behaviour is learned more implicitly than if a cognitive secondary 
task was not performed concurrendy [8] . 

However, not all intervention strategies are used unambiguously 
with respect to the implicit/ explicit distinction. For many motor 
learning strategies, it seems unclear whether they promote implicit 
or explicit motor learning or whether their ability to promote 
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Table 2. Characteristics of the expert panel. 



Category 



Subcategory 



Results (absolute numbers) 



Gender 



Male 
Female 



Working country 



In last 5 years mainly worked as 



Background* 



Expert in which motor learning area* 



Target population working with* 



Years of experiences 



>70: 

Not wanted to state/missing: 

England/UK: 

The Netherlands: 

USA: 

Australia: 

Canada: 

France: 

Belgium: 

Germany: 

China/Hong Kong: 

New Zealand: 

Switzerland: 

Missing: 

Researcher: 

Lecturer/Educator: 

Therapist: 

Both researcher and lecturer equally: 
Other (e.g., consultant, psychologist): 
Missing: 

Rehabilitation Practitioner (PT, OT, ST # ): 

Movement Scientist: 

Psychologist: 

Coach: 

Other (e.g., biomechanist, sport scientist): 

Rehabilitation: 

Sports: 

Fundamental research (neuroscience): 

Elderly: 

Children: 

Education: 

Other (e.g., cognitive psychology, mental health): 

Neurological patients (adults): 

Elderly: 

Healthy population in general: 
Athletes: 

Neurological patients (children): 
Orthopaedic patients (adults): 
Healthy children: 

Other (e.g., therapists, patients with mental health problems): 

Research 

Not applicable 

Practice 



23 
27 



Age category 


20- 


-30: 


3 




30- 


-40: 


12 




40- 


-50: 


19 




50- 


-60: 


14 




60- 


-70: 


2 



2 
3 

16 

8 

6 

4 

4 

2 

2 

2 

1/2 
1 
1 
1 

22 

8 

11 

2 

6 

1 

25 

18 

11 

8 

5 

35 

18 

13 

9 

4 

2 

4 

23 

14 

12 

11 

8 

1 

1 

5 

Mean: 14.1 (SD: 11.8) 
7 

Mean: 11.8 (SD: 10.0) 



PLOS ONE | www.plosone.org 



4 



June 2014 | Volume 9 | Issue 6 | e100227 



Delphi Technique on Implicit and Explicit Motor Learning 



Table 2. Cont. 





Category 


Subcategory 


Results (absolute numbers) 




Not applicable 


10 



*:more answer options were possible;^PT: Physiotherapist, OT: Occupational Therapist, ST: Speech and Language Therapist; Table is based on data of n = 50 experts 
(n = 49 experts completed Round 1 and 2; n = 1 expert completed Round 2 only); 
doi:1 0.1 371 /journal.pone.01 00227.t002 



either form of learning is a function of the target population or the 
specific learning context in which they are applied. For example, 
discovery learning is regarded by some researchers as likely to 
result in predominantly implicit learning outcomes, whereas, other 
researchers argue that predominandy explicit outcomes result 
[23]. Yet, trial and error learning, which in practice seem little 
different from discovery learning, has been described to promote 
explicit motor learning [19,24]. 

For therapists, coaches, researchers and teachers, uniform 
terminology is particularly important. Effective transfer of research 
results to clinical practice and education is promoted by clear 
terminology, and allows therapists and coaches to speak a 
common language among themselves (e.g., to set up treatment 
plans) or when instructing students [25]. The aim of this study was 
therefore to seek consensus regarding the definitions, descriptions 
and classification of terms related to the general distinction 
between implicit and explicit forms of motor learning. 

Method 

A survey consisting of a series of sequential rounds interspersed 
by controlled feedback [26] was performed to collect and 
aggregate informed judgments about motor learning from a group 

Table 3. Definition of explicit motor learning. 



of experts. The survey consisted of three rounds, which were 
designed and distributed using an online survey programme 
(SurveyMonkey Inc, SurveyMonkey.com, California, USA). More 
detailed information about the method and rationale for the entire 
survey is presented elsewhere [27]. 

A Delphi technique was embedded into the first two rounds of 
the survey to seek consensus regarding definitions, descriptions 
and classifications related to the explicit/implicit distinction in 
motor learning. Although there is minimal scientific evidence 
available to inform decisions about the number of survey rounds 
appropriate for a Delphi technique, two or three rounds have 
typically been employed [28]. Information regarding the content 
and the results of the third round is not presented here as this 
round was not used as part of the Delphi technique. 

The Central Ethics Committee Atrium-Orbis-Zuyd (Institu- 
tional Review Board) was contacted and formal written permission 
to perform the study described in the protocol [26] was obtained 
(13-N-144). The study was excluded from IRB review, because 
under the law, Medically Scientific Research with people (WMO), 
it does not submit people to actions or impose specific behaviors 
on them. 



Explicit Motor Learning 



Round 1 (n = 49) 
Attributes provided in 
first round and 
percentages chosen 



Round 2 (n = 44) 

Definition provided in second round 

% agreement 
Comments after second 
round 

(n = agreed/disagreed)* 



•Involves cognitive stages:*85.7% 

•Generates verbal knowledge of movement performance (e.g., facts and rules): 79.6% 
•Dependent on working memory involvement: 73.5% 

■Facilitated by instructions about how to perform the movement: 67.3% 

•With intention to learn: 63.3% 

•With purposeful hypothesis testing: 42.9% 

■Learning processes are faster (compared to implicit}: 22.4% 

•Other: 12.2% 

Explicit motor learning can be defined as learning which generates verbal knowledge of movement 
performance (e.g., facts and rules), involves cognitive stages within the learning process and is dependent 
on working memory involvement 

95.5% 

•Aspect of an 'internal focus' should be involved (n = 0/1) 
•Three key attributes in definition are related and therefore redundant (n = 1/0) 
•Disagreement about the involvement of cognitive stages (n = 2/0)/working memory (n = 2/0) 
•Disagreement about the distinction between implicit and explicit learning in general (n = 1/0) 
•Disagreement about the verbal/explicit instructions (n = 2/0) 



*Attributes in bold were taken into account for the definition in Round 2; #:Comments of experts who did not agree are underlined. Numbers in brackets signify 
amount of times that this comment was provided by experts who agreed/disagreed with definition. 
doi:1 0.1 371 /journal.pone.01 00227.t003 
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Table 4. Definition of implicit motor learning. 



Implicit Motor Learning 



Round 1 (n = 49) 
Attributes provided in 
first round and 
percentages chosen 



Round 2 (n = 44) 

Definition provided in second round 

% agreement 
Comments after second 
round 

(n = agreed/disagreed) ' 



•No or minimal increase in verbal knowledge*: 81.6% 

•Skills are (unconsciously) retrieved from implicit memory: 81.6% 

•Skills are learned without awareness: 69. 4% 

•Without exposure to verbal instructions about how to perform: 53.1% 
•Robust to disruption: 42.9% 

•Learning process takes longer (compared to explicit}: 42.9% 
•Without an initial cognitive stage: 36.7% 
•No purposeful hypothesis testing: 30.6% 
•Other: 16.3% 

Implicit motor learning can be defined as learning which progresses with no or minimal increase in verbal 
knowledge of movement performance (e.g., facts and rules) and without awareness. Implicitly learned 
skills are (unconsciously) retrieved from implicit memory. 

88.6% 

■Disagreement about the fact that skills are learned without awareness (n = 7/2) 
■Disagreement about use of the term "implicit memory" (n = 5/1) 
•Same attributes should be used in definition of implicit and explicit (n = 1/0} 
•Definition contains assumptions that should be tested first (n = 0/1) 

•Definitions should take the complexity of cognitive involvement more into account (n = 1/0) 
■External focus should be involved (n = 1/1) 



*Attributes in bold were taken into account for the definition in Round 2. #: Comments of experts who did not agree are underlined. Numbers in brackets signify 
amount of times that this comment was provided by experts who agreed/disagreed with definition. 
doi:1 0.1 371 /joumal.pone.01 00227.t004 



Procedure 

A referee group consisting of seven researchers with back- 
grounds in epidemiology, physiotherapy, occupational therapy, 
movement sciences and psychology supervised and monitored the 
process. The group conducted the literature search, identified 
experts to be approached to complete the surveys, and prepared 
the questions for each survey round. Between each survey round, 
the group performed a preliminary analysis of data blinded to the 
identity of the experts. In addition, two members of the referee 



group (MK, SB) were responsible for distributing and monitoring 
the survey (e.g., sending reminders and feedback reports). 

A panel of international experts was invited to contribute to the 
study. Panel members were initially selected on the basis of 
literature search or the networks of the referee group. Criteria for 
selection of an expert were based on either scientific publication(s) 
in the field of motor learning (researcher) or at least three years of 
working experience applying motor learning in practice plus 
involvement in education or research (therapist, coach, lecturer). 



Table 5. Percentage of experts who knew/used the provided strategies. 





Strategy* 


Percentage of experts who knew the 
strategy 


Percentages of experts who have used the strategy before in research 
or practice 


Trial and error 


91.8% 


73.5% 


Observational 


89.8% 


67.3% 


Errorless 


89.8% 


63.3% 


Movement imagery 


85.7% 


40.8% 


Discovery 


77.6% 


36.7% 


Dual task 


77.6% 


57.1% 


Analogy 


73.9% 


55.1% 


Incidental 


65.3% 




Self-regulatory 


49.0% 




Constraints-led approach 


46.7% 





*Strategies in bold were taken into account in Round 2. 
doi:1 0.1 371/joumal.pone.0100227.t005 
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Table 6. Description of the best known strategies. 



Round 1 (n = 49) Round 2 (n = 43/44)" 

Description provided % Adapted description in 

in first round agreement Comments* second round % agreement 



Trial and error 
learning 


Learning by repeatedly 
attempting to perform 
a task during which 
errors are detected 
and corrected. 


71.4% 


The learner must (be able) 
to detect the error (n = 6); 

Learning is an iterative process 
(n = 1); Correction of errors 
should not be emphasised (n = 1) 


Learning by repeatedly 
attempting to perform a task 
during which the learner detects 
errors and corrects them. 


84.1% agreed; 9.1% 
preferred description 
from Round 1 


Observational 
learning 


Learning by observing 
a movement. The 
observer determines the 
key spatial and/or 
temporal features of 
the task through 
observation, thereby 
creating a cognitive 
representation of the 
action pattern. 


69.4% 


Unsure about/delete 
"cognitive representation" 
(n = 7); The demonstrator/ 
therapist can also direct the 
learner to the key features 
(n = 1) 


Learning by observing a 
movement. The observer 
determines the key spatial 
and/or temporal features of the 
task through observation, 
and/or is directed to these 
features by the demonstrator/ 
therapist. 


68.2% agreed; 20.5% 
preferred description 
from Round 1 


Errorless 
learning 


Learning facilitated by 
constraining the 
learning environment 
so that very few errors 
occur. 


67.3% 


Learning environment and 
the instructions and skill 
difficulty can be constrained 
as part of the learning 
environment (n = 3); Should be 
applied particularly in early 
phase of learning (n= 1}; Replace 
"very few errors" with 
"no errors" (n = 1) 


Learning facilitated by 
constraining the learning 
environment (e.g., instructions, 
skill difficulty) so that very few 
errors occur. 


77.3% agreed;13.6% 
preferred description 
from Round 1 


Movement 
Imagery 


Learning by imagining 
oneself undertaking the 
skilled movement 
without actually doing 
the movement. 


71.4% 


Imagery should be from the 
first person perspective 
(n = 2); Exchange 
"undertaking" with 
"performing" (n = 1); 
Suggestions for terming the 
strategy (mental rehearsal, 
motor imagery) (n = 2) 


Learning by imagining oneself 
performing the skilled movement 
(in the first or third person 
perspective) without actually 
physically performing the 
movement. 


81.8% agreed; 13.6% 
preferred description 
from Round 1 


Discovery 
learning 


Learning without 
guidance, instructions 
or feedback from 
another person. 


57.1% 


Without information from 
other sources (book, website) 

(n = 2); It is necessary to give 
instructions or feedback (n = 2); 


Learning without guidance or 
feedback from another person 
or information source. 


75.0% agreed; 18.2% 
preferred description 
from Round 1 



Learning is facilitated by 
(constrained) context (n = 3); Use 
(pure) discovery learning as a 
synonym for Trial and error 
(use this description) (n = l) 



Dual-task 
learning 


Learning of a skill 
during simultaneous 
performance of another 
skill. The secondary 
task can be a motor 
or cognitive task. 


61.2% 


The (second) task must be 
of equal importance/difficulty 
and attention demanding 

(n = 5); Doubts about whether 
dual task is a form of learning 

(n = 3) + 


Learning of a skill while 
simultaneously performing 
another task. The second task 
can be a motor or cognitive 
task but must be attention 
demanding 


81.8% agreed; 9.1% 
preferred description 
from round one 


Analogy 
learning 


Learning facilitated by 
metaphors. The complex 
structure of the 
to-be-learned skill is 
integrated in a simple 


51.0% 


Did not agree with term 
'biomechanical' (n = 6) 


Learning facilitated by metaphors. 
The complex structure of the 
to-be-learned skill is integrated 
into a simple metaphor that the 
learner is provided with. 


95.5% agreed; 2.3% 
preferred description 
from round one 



biomechanical metaphor 
that the learner is 
provided with 



"Comments in bold were taken into account for the adapted description; # One expert did not complete all questions; ^This remark was taken into account in separate 
questions in Round 2 (results not presented). 
doi:1 0.1 371 /joumal.pone.01 00227.t006 



In a preliminary recruitment round, eligible experts were 
invited by mail to participate in the study. They were given a 
comprehensive introduction to the aims and content of the survey 
rounds, informed of the expected amount of time necessary to 
complete each survey and provided with a personal link to the 



online survey program. After informed consent was obtained from 
the experts, they were asked to provide personal information (e.g., 
background, years of experience, special interests). Those experts 
who were invited to participate were asked to recommend other 
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experts in the field (so-called snowball sampling method), who 
were subsequendy contacted in the same manner. 

No clear guidelines regarding the optimal panel size for a 
Delphi study exist [29]. Consistent with another study using a 
Delphi technique [30], a minimum panel size of 30 experts was 
targeted, comprising approximately ten motor learning research- 
ers in rehabilitation, ten in healthy individuals and sports and ten 
with experience applying motor learning in daily practice. 

Content of the survey rounds and analysis. Within the 
survey rounds, we distinguished between definitions and descrip- 
tions. The term "definition" was used when referring to forms of 
learning (e.g., implicit, explicit), whereas the term "description" 
was used when referring to motor learning strategies (e.g., errorless 
learning, trial and error learning). We made this distinction 
because the term 'definition' implies theoretical attributes/features 
of learning, while the term 'description' implies elements of how a 
strategy is applied. 

Additionally, implicit and explicit motor learning have been 
described as representing a dichotomy in learning and also as 
representing tail ends of a learning continuum, so for the 
categorisation of intervention strategies we used answering 
categories that left room for both perspectives. 

Table 1 presents the content of the survey rounds. Each survey 
round was divided into two parts. The first part of Round 1 
focussed on creating a basic definition of implicit and explicit 
motor learning. The second part focussed on identifying, 
describing and classifying learning strategies. Questions in part 
one of Round 2 and 3 were used to verify responses in the first 
round and second round respectively and to elaborate issues 
identified by the expert panel. Questions in part two of Round 2 
and 3 addressed other predefined topics (results are not presented 
in this article). 

In preparation of the Delphi, the referee group performed a 
literature search in different fields of motor learning (sports, 
rehabilitation, fundamental research) using both scientific research 
articles and grey literature. The group identified several search 
terms, implicit, explicit, motor learning, skill acquisition, and used 
MeSH terms when possible. From these resources, e.g. 
[1,8,11,17,31-35], statements which were related to implicit and 
explicit motor learning were extracted and compared. The referee 
group tried to improve readability of the statements by using 
comparable formulation. 

Analysis of the data was conducted blind to the names and 
characteristics of the expert respondents. Open comments and 
additions made by the experts were clustered in themes and 
carefully considered by the referee group. Consistent with other 
studies, consensus was considered to have been reached when & 
70% of the experts agreed on a certain topic [30,36,37]. 
Differences in values and beliefs within the different professions 
represented by the experts might have influenced the results, so in 
cases where &70% of the experts agreed, the referee group 
checked for a profession-based imbalance in the responses, which 
was not the case. 

If consensus was achieved, final definitions and descriptions 
were formulated. If no consensus was achieved, the topic and 
answers were presented to the expert panel in the following round. 

After each round, panel members received a feedback report 
that summarised the response percentages for each question, as 
well as responses to open questions and additional comments. In 
these feedback reports, the results were clustered but not analysed 
or interpreted. 
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Table 8. Examples of the best known strategies provided by the experts. 



Strategy Two random selected examples provided by experts 



Trial and error learning "Structure the learning environment so that errors will be made, but a positive outcome is achievable. Inform the learner that following the 
practice session they will be asked to describe the different techniques they tried and list what worked and what didn't." 



Putting on a jumper: "Prompt when needed to avoid frustration but encourage patient to do without help. Positive reinforcement. Requires good 
attention levels." 


Observational learning 


"Demonstration is probably used quite frequently by therapists who wish to demonstrate what they want a patient to do, or how they want 
them to do it. In my experience, this is generally accompanied by verbal instructions, making it more explicit. Patients may observe each other in 
a group setting, which could be formally set up (working in pairs) to create an observational learning environment -for example, for performing 
balance tasks."" 


"This technique is frequently used in dance classes where one dancer acts as a model and the other observe and then imitate." 


Errorless learning 


"In aikido, novices may learn new techniques with a more experienced partner that would help novices to succeed every time they perform it.' 


"Learning to walk after a stroke with body weight support and a treadmill, and gradually increasing the body weight the person is taking as well 
as the treadmill speed." 


Movement Imagery 


"With patients who are physically unable to perform such a movement at the beginning of rehabilitation, or if they fatigue quickly during 
physical practice." 


"Imaging oneself climbing a wall and then climbing it." 


Discovery learning 


"Children in a playful setting discover biomechanics of building with blocks." 


"For teaching previously unknown skill - e.g., making piece of toast one handed. Explain what is needed and leave patient to work out how. 
Would need high level problem solving including attention and memory. Avoid distraction. Would require positive reinforcement." 


Dual task learning 


"Having a child count backwards by 2's (depending on age and cognitive level) while walking on the balance beam." 


"Clinicians working on more complex or real-world environments where motor tasks are combined with other motor tasks or cognitive tasks 
(such as talking). Instructions can be used to prioritize a task or it can be left to the discretion of the performer. Feedback and measures of 
performance should be provided on both tasks." 


Analogy learning 


"Jumping pattern: "reach for an apple up in the tree" "Basketball shot: "putting your hand into the cookie jar." 



"Dance tango (in particular how to provide a good abrazo): like maintaining a newspaper always opened." 



doi:1 0.1 371 /journal.pone.01 00227.t008 



Results 

The recruitment process is shown in Figure 1. In total, 155 
experts were invited to participate. Thirty-nine experts agreed to 
participate initially and recommended a further 49 experts. Forty- 
nine experts completed the Round 1 survey and 44 completed the 
Round 2 survey. Characteristics of these experts are shown in 
Table 2. Experts were heterogeneous with regard to age, 
background and current working situation. Although the expert 
panel was internationally diverse, most were based in Europe. Of 
the 1 1 experts who did not respond to invitations or reminders, 
only two reported lack of time as the reason for non-response. 

Definitions of implicit and explicit motor learning 

The results with regard to the definitions of explicit and implicit 
motor learning are shown in Tables 3 and 4. Of the experts, 
95.5% agreed in general with the following definition of explicit 
motor learning: learning which generates verbal knowledge of movement 
performance (e.g., facts and rules), involves cognitive stages within the learning 
process and is dependent on working memory involvement. 

Six experts (12.2%) proposed additional attributes of the 
definition of explicit learning that were not provided initially 
(category "other"). The additions included practical attributes 
(e.g., "using an internal focus of attention", "using feedback") and 
attributes related to the result of explicit learning (e.g., "responsible 
for rapid early improvement'), but they were not incorporated into the 
definition as each was only suggested by one or two experts. 

Of the experts, 88.6% agreed in general with the following 
definition of implicit motor learning: learning which progresses 
with no or minimal increase in verbal knowledge of movement 
performance (e.g., facts and rules) and without awareness. 



Implicitly learned skills are (unconsciously) retrieved from implicit 
memory. 

Eight experts (16.3%) proposed additional attributes of the 
definition of implicit motor learning, which were eventually not 
incorporated into the definition, despite their importance for 
discussion (e.g., an "external focus of attention" is used; the learning is 
"goal orientated"; relies on "functional practice in a meaningful 
environment"). One expert pointed out that implicit learning is 
"not non-cognitive" and "not unconscious" but rather "non-verbal . This 
expert further pointed out that implicit learning "often does involve 
awareness of trying to accomplish something" . 

Thirteen existing definitions or extractions from literature were 
preferred or deemed to be as good as the new definitions, but none 
were mentioned more than once. 

Descriptions of strategies 

Table 5 provides an overview of 1 0 motor learning intervention 
strategies that were identified from the literature and gives the 
percentage of experts who indicated that they were aware of the 
strategies and/or used the strategies. Seven strategies were known 
by more than 70% of the experts and were therefore included in 
Round 2. After modification of the strategy descriptions, based on 
the comments in Round 1, consensus was achieved for six of the 
seven descriptions (see Table 6). Only for observational learning did 
percentage agreement decrease slighdy after the description was 
reformulated (from 69.4% to 68.2%). 

Classification of strategies 

Responses regarding classification of whether the strategies are 
likely to result in (more) implicit or (more) explicit forms of motor 
learning were diverse (see Table 7). 
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Round 1 suggested that none of the strategies can be 
categorized as promoting just one form of motor learning. For 
the errorless, dual-task and analogy learning strategies, there was 
however a slight trend for the experts to consider these strategies as 
likely to result in a more implicit form of learning. Depending on 
the strategy, between 1 and 5 experts did not classify the separate 
strategies into one of the provided categories, but chose the option 
'other' (between 4.5-16.7% of the sample). 

A common argument in the open comment box was that the 
strategy could promote both implicit and explicit motor learning. 
Factors, such as, instructions, constraints in the environment, type 
of task/ skill and the abilities of the learner were all deemed to have 
an influence on the outcomes of the learning strategy. According 
to the experts, manipulation of these factors has a profound 
influence over the degree to which a strategy results in implicit or 
explicit motor learning. 

Additional strategies 

Twenty-two alternative motor learning strategies were suggest- 
ed, which were not included in the initial list presented to the 
experts (e.g., win shift lose stay, verbal overshadowing, blocked 
practice, applied behaviour analysis). None of these strategies were 
mentioned by more than one expert and were therefore not 
incorporated in the following surveys rounds. Other suggestions 
were related more generally to the focus of attention during 
learning, the provision of feedback, the repetition and variability of 
practice and manual facilitation. 

As a result of the diversity in answers and additional statements 
made by the experts, the referee group decided not to strive to seek 
consensus with regard to the classification but rather to explore the 
reasons for diversity. This was done in the third survey round and 
resulted in an overview of practical experiences, opinions and 
verifications of statements (results not presented). 

Examples of the application of the strategies in clinical 
practice 

The number of examples of the application of different 
strategies in clinical practice ranged from 30 (discovery learning) 
to 41 (errorless learning). For each strategy, the referee group 
chose two examples to present in this article (Table 8). 

Discussion 

The aim of this study was to seek consensus on the definitions 
and descriptions of terms related to the conceptual distinction 
between implicit and explicit motor learning. Within a heteroge- 
neous international group of experts, consensus regarding 
definitions of implicit and explicit motor learning, and descriptions 
of the best known strategies used in the context of implicit and 
explicit motor learning, was reached. Both definitions incorporate 
central aspects of motor learning (e.g., form of memory and type of 
knowledge). Incorporation of more than one central aspect of 
motor learning is preferable to oudining a single aspect, as 
sometimes occurs in definitions, and suggests that there was at least 
some degree of consensus by the experts. 

Consensus suggests that experts from the different fields 
represented within the study may think about and describe motor 
learning and the underlying processes in a comparable way, at 
least at a more theoretical level. 

According to the responses of the experts in this study analogy 
learning, errorless learning and dual task learning seem to promote 
more implicit learning in general. However, no consensus was 
reached within the expert panel on the classification of motor 
learning strategies for promoting a (more) implicit or (more) 



explicit form of motor learning. Based on the results of this study, 
it seems that most intervention strategies do not naturally promote 
implicit or explicit motor learning. They can promote either form 
of learning depending on their use in a specific learning situation 
and/or target population. It is probably impossible, and perhaps 
not even desirable, to achieve consensus. This result might be a 
consequence of the complexity of applying motor learning 
strategies in everyday practice. For example, athletes or patients 
usually need a tailored approach and the application of learning 
strategies is determined by multiple factors. Consequently, for 
research and education it is even more important that the 
application of an intervention strategy in a specific context is 
always described in detail. 

Critical reflection on the study and the study results 

To our knowledge, this is the first study that uses a Delphi 
technique in the field of motor learning. The results generated and 
summarized within the study are based on knowledge, opinions 
and practical experiences of an international expert panel. 
Consequendy, the results should be interpreted tentatively; an 
expert group's opinion rather than empirical evidence. 

Importandy, although consensus was obtained regarding the 
definitions, this does not mean that all of the experts, or indeed the 
authors (referee group), agree with the final definitions and 
descriptions. For instance, the final definition of implicit motor 
learning suggests that learning progresses "without awareness", 
but there are clearly occasions (e.g., sport, rehabilitation) when a 
person has intention to learn and is aware of learning, especially 
when outcome feedback is readily available [21,38]. It also seems 
unlikely that learning ever progresses with 'no' increase in verbal 
knowledge. 

Although the Delphi technique is a well-accepted method for 
investigating opinions, there is currently no agreement on the 
meaning of consensus [28] . In our study, consensus was regarded as 
agreement within a selected group of leading experts on a certain 
topic, based on a criterion of 70% agreement or greater. However, 
lack of consensus (i.e., less than 70% agreement) does not directiy 
imply that a statement was invalid, but may suggest that more 
plausible possibilities exist or that no alternatives exist yet. 
Numerous comments and statements were made by individual 
experts in response to the open questions. Although all were 
distributed in the feedback report, most were not carried back into 
the survey. As the aim of the study was to achieve consensus (a 
quantitative approach), we unfortunately were not able to take all 
single statements into account (a more qualitative approach). 
Consequendy, comments/statements which other experts may 
have agreed upon might have been overlooked. 

The quality of the findings from a Delphi study is strongly 
related to the heterogeneity and representativeness of the expert 
panel. Although the response rate in the current study was low, the 
experts who participated can be described as heterogeneous with 
regard to their backgrounds, special interest and working 
experience. Further, the different practical areas of motor learning 
are represented by the expert panel. We tried to overcome 
selection bias within the sample by using snow-ball sampling; 
nevertheless, some selection bias may have occurred, as most of 
the experts who participated were based in Europe. This might be 
explained by the fact that six of the seven referee group members, 
whose networks were used to identify experts, were also based in 
Europe. We do not know to what extent the origin and 
background of the experts influenced the results, so we acknowl- 
edge that cultural values may account for some of our findings 
(especially, lack of consensus). 
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Contribution to scientific literature and implications for 
research 

Despite the limitations already discussed, we believe that it is 
important to use uniform terminology when describing the content 
of motor learning studies and practical sessions, and within 
education. The study is a first important step towards helping 
therapists, researchers and other professionals to communicate 
about motor learning in general and to distinguish fundamentally 
between implicit and explicit motor learning more specifically. 
The added value of the study is that the definitions and 
descriptions that emerged are based on the opinions of an expert 
panel from different fields of motor learning, which might help to 
promote a common language across different fields. 

Future applied research is needed to confirm the findings. 
Underlying neurophysiological and behavioural aspects of the 
definitions should be investigated by fundamental research. 
Clinical research investigating clearly defined and described 
techniques is needed to investigate whether the definitions of 
implicit and explicit motor learning, as well as the descriptions of 
the strategies, are feasible and applicable within clinical practice. 
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